Nowadays, there is an increasing use of big data systems. The term “big data” as used herein throughout the specification and claims is used to denote a term encompassing any collection of data sets so large and complex that it becomes difficult to process the information by using traditional data processing applications. The challenges include among others analysis, capture, search, sharing, storage, transfer, etc. The trend to use larger data sets results from the fact that the additional information derivable from analysis of a single large set of related data, as compared to separate smaller sets with the same total amount of data, allows finding correlations to spot business trends, and the like.
The use of big data systems regularly encounter limitations due to the large data sets used. These limitations also affect Internet search, finance and business informatics. Data sets grow in size in part because they are increasingly being gathered by ubiquitous information-sensing mobile devices, software logs, cameras, microphones, radio-frequency identification readers (RFID), and wireless networks. The world's technological per-capita capacity to store information has roughly doubled every 40 months since the 1980's as of 2012.
Big data systems is difficult to work with using most relational database management systems and desktop statistics and visualization packages, which require instead massive parallel software running on tens, hundreds, or even thousands of servers. The term “big data” varies depending on the capabilities of the organization managing the set, and on the capabilities of the applications that are traditionally used to process and analyze the data set in its domain.
In addition, different departments in various organizations such as marketing, business development, strategic planning, etc. need to utilize information that is available to them in order to better understand and analyze expected trends in their business and how to promote their business. Typically, the information is available both as internal information (stored at the organization's database) and from external sources. Typical tools currently available, are, platforms to enable big data mining, Data Warehouse (DWH), various Business Intelligence (BI) tools, and the like. However, these tools are infrastructural tools, which are typically used by IT people, but are not adapted to enable in depth investigation nor to provide conclusions that can be used by marketing people or business development people to improve their business.
Some of the drawbacks associated with the currently available solutions are:
1. They are not suitable to identify the business problems. More specifically, they are able to provide out of the box concrete answers to specific business problems in an actionable manner and are typically concerned with processing non-aggregated data. In order to overcome at least partially these disadvantages, the person who seeks the answers needs to draft the proper queries that relate to the existing products, the available technologies, etc., and then to hand them over to the BI department so that they can be converted into queries that fit the interface with the available data mining software and then collect the relevant data, that can be followed by building a model that would provide the required information.2. The existing solutions do not include a continuous update of the models, a fact that leads to a situation where the models might become irrelevant for marketing use within a too short period of time.3. Longer time to market periods due to the inability of the marketing people to promptly react to changes that occur in the dynamic market at which they operate.4. Most of the existing solutions rely on information retrieved from specific information sources at the organization and do not take into account information that is available from additional external sources.
The changes which the telecommunication market has been experiencing in last years, create a lot of pressure on the various companies to find new sources for income and to do so, they need to rely on one of their assets, namely their customers and the vast amount of information that has been gathered within the company on their activities.